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he authors conducted a qualitative meta-analysis of the research on 
assessment accommodations for students who are deaf or hard of hear- 
ing. There were 16 identified studies that analyzed the impact of factors 
related to student performance on academic assessments across differ- 
ent educational settings, content areas, and types of assessment accom- 
modations. The meta-analysis found that the results of analyses of group 
effects of accommodated versus unaccommodated test formats are often 
not significant, test-level factors exist that can affect how students per- 
ceive the assessments, and differences exist in how test items function 
across different conditions. Student-level factors, including educational 
context and academic proficiency, influence accommodations’ role in 
assessment processes. The results of this analysis highlight the complex- 
ity of and intersections between student-level factors, test-level factors, 
and larger policy contexts. Findings are discussed within the context of 
larger changes in academic assessment, including computer-based 
administration and high-stakes testing. 
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Educational testing in U.S. public edu- 
cation continues to shift toward two 
goals: (a) standardized measurement 
of student outcomes and (b) inclusive 
assessment participation policies. The 
first goal allows for comparisons of 
student progress across different set- 
tings. The scope of comparisons varies 
widely: Individual schools in a state can 
be compared, or, per one of the goals 
of the upcoming Common Core Stan- 


dards Assessments, individual states 
across the nation (Christensen, Lazarus, 
Crone, & Thurlow, 2008; Common Core 
State Standards Initiative, 2010; Rig- 
ney, Wiley, & Kopriva, 2008; Thurlow, 
Lazarus, Thompson, & Robey, 2002). 
At the same time, inclusive assessment 
participation policies require that all 
students be provided with an opportu- 
nity to demonstrate their knowledge 
and skills on these tests. Expansion of 
the set of participants in an assessment 
can be challenging to implement in a 
fair and valid way when tests were not 
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originally designed to be accessible to 
all students, particularly students with 
disabilities or those who are English 
Language Learners (Cawthon, Ho, 
Patel, Potvin & Trundt, 2009; Kopriva, 
Emick, Hipolito-Delgado, & Cameron, 
2007; Shaft el, Belton-Kocher, Glass- 
nap, & Poggio, 2006; Sireci, Li, & 
Scarpati, 2003). 

In the United States, both state-level 
and federal education policies (Indi- 
viduals With Disabilities Education 
Improvement Act, 2004; Americans 
With Disabilities Act, 1990) govern 
how students with disabilities partici- 
pate in large-scale assessments. Within 
the context of accountability reforms 
such as NCLB, tests are high-stakes 
activities because the resultant scores 
drive decisions about student promo- 
tion and school accountability report 
cards, and in some states, teacher com- 
pensation. The impact of decisions 
that include consideration of standard- 
ized assessments thus reaches across 
all levels of the education system, pro- 
viding a significant impetus to facilitate 
inclusive participation of diverse stu- 
dent populations in a way that yields 
results that are both meaningful and 
valid (Elliott & Roach, 2007; Fairbain 
& Fox, 2009; Kopriva, 2008; Liu & An- 
derson, 2008; Shaftel, Yang, Glassnap, 
& Poggio, 2006). 

Assessment accommodations are 
one strategy frequently used to expand 
the inclusivity of standardized assess- 
ments for students with disabilities 
(Lazarus, Thurlow, Lail, Eisenbraun, & 
Kato, 2006). Common accommoda- 
tions allowed on state standardized 
assessments include extended time, a 
separate room for administration of 
the test, having test items read aloud 
(for tests that are not in a language in 
which a student reads proficiently), 
and having test instructions read aloud 
before the student begins the assess- 
ment (Christensen, Braam, Scullin, & 
Thurlow, 2011). Students may receive 


test accommodations singly, or to- 
gether as a package. In contrast with 
test modifications, in which changes 
to the test format or content may alter 
the construct being measured, assess- 
ment accommodations are meant to 
increase access to the test content 
while allowing for the score to be inter- 
preted in the same manner as that for 
a test taken without an accommoda- 
tion (American Educational Research 
Association, American Psychological 
Association, & National Council on 
Measurement in Education, 1999). 
Maintaining the construct of the test 
item, and thus the validity of the test 
score, is critical when scores are being 
used for high-stakes decisions such as 
those derived from NCLB. 

While state assessment policies 
seek to offer clear guidance on the use 
of accommodations in high-stakes 
assessments, the body of empirical re- 
search literature examining the actual 
effects of test accommodations on stu- 
dent test scores offers less clarity as to 
accommodations’ potential impact on 
test score validity (Abedi, Hofstetter, & 
Lord, 2004; Bolt & Thurlow, 2004). 
While there are examples of test 
accommodations having the desired 
effect of providing access without 
changing test difficulty (Fletcher et al., 
2006; Schulte, Elliott, & Kratochwill, 
2001), there are also examples of test 
accommodations that have the po - 
tential to change the construct of the 
test item (Crawford & Tindal, 2004; 
Fletcher et al., 2006), and that some- 
times even make the test item more 
difficult for the student using the 
accommodation (Sireci, Scarpati, & Li, 
2005). 

Comprehensive reviews of research 
findings on the effects of test accom- 
modations on student scores are avail- 
able for students with disabilities as a 
whole (Bolt & Thurlow, 2004; Towles- 
Reeves, Kleinert, & Muhomba, 2009; 
Zenisky & Sireci, 2007), and for English 


Language Learners (Kopriva, 2008, pp. 
81-100; Shaftel et al., 2005), as well as 
in disaggregated form, by type of 
accommodation (e.g., extended time, 
Elliott & Marquart, 2004; read-aloud, 
Meloy, Deville, & Frisbie, 2002). Com- 
mon to all these studies is an appreci- 
ation of the importance of knowing 
the content area of the test as well as 
the student’s proficiency in that con- 
tent area when efforts are being made 
to understand the potential effect of 
an accommodation in making a test 
more accessible for the student (e.g., 
Bielinski, Ysseldyke, Bolt, J. Friede- 
bach, & M. Friedebach, 2001; Bolt & 
Thurlow, 2006; Mandinach, Bridge- 
man, Cahalan-Laitusis, & Tripani, 2005). 
The reviews cited above provide excel- 
lent discussions of the nuances of the 
findings, the places where research 
findings can reasonably be translated 
into assessment policy, and the chal- 
lenge of fitting the accommodations to 
the test content area as well as the stu- 
dent’s individual needs. 

For students in high-incidence dis- 
ability populations, such as students 
who have a learning disability or atten- 
tion deficit hyperactivity disorder, 
many of the findings in the extant 
reviews offer sufficient understanding 
of how the research literature can 
inform practice (Cawthon, Kaye, Lock- 
hart, & Beretvas, 2012 ; Keiffer, Lesaux, 
Rivera, & Francis, 2009). However, in 
research on “students with disabilities” 
as a whole, students with low-inci- 
dence disabilities are aggregated into a 
larger group, which reduces the capac- 
ity to identify how an accommodation 
interacts with their characteristics, 
specifically. Diffusing findings across 
diverse student groups goes against 
one of the findings of the held: that an 
effective accommodation, meaning 
one that facilitates access to test con- 
tent yet maintains the construct of the 
test, requires consideration of the 
characteristics of the student, the test, 
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and the accommodation (Cawthon, 
2009). There are studies that specifi- 
cally focus on a subgroup of students 
with disabilities, such as those with 
more severe cognitive disabilities, 
allowing for the kind of discussion that 
can meaningfully translate into assess- 
ment practice guidelines for these 
populations (e.g., Baker, 2010; Elliott & 
Roach, 2007; Zalta & Pullin, 2004). If 
research findings on accommodations 
are to be meaningfully interpreted, 
accommodations’ effects within indi- 
vidual disability groups need to be con- 
sidered in their own right (Sireci et al. , 

2005) . 

The present article focuses on the 
research literature on the effects of 
assessment accommodations used 
with students who are deaf or hard of 
hearing, a population that makes up 
less than 1 % of students who receive 
services under the Individuals With 
Disabilities Education Act (see http:// 
www.ideadata.org/partbchildcount.asp, 
a website of the Data Accountability 
Center). Students who are deaf or hard 
of hearing come to educational assess- 
ment with a unique but critically 
important set of personal characteris- 
tics and learning experiences (Caw- 
thon, 2007). Unfortunately, much of 
the accommodations research litera- 
ture on these students does not take 
into account the demographics, such 
as degree of hearing loss or language 
use, in the study designs (Cawthon, 
2007; Cawthon & Online Research Lab, 

2006) . To continue the argument from 
above, although it may be tempting to 
group the members of this low-inci- 
dence student population together, 
deaf and hard of hearing students are 
diverse in their linguistic, communica- 
tion, and cultural identification (R E. 
Spencer & Marschark, 2010). There is 
no “typical” deaf or hard of hearing stu- 
dent demographic (Mitchell, 2004): 
Some students come to primary edu- 
cation fluent in American Sign Lan- 


guage (ASL), others use spoken lan- 
guage, others a combination of visual 
and spoken modalities (Emmory, Pet- 
rich, & Gollan, 2013; Singleton, Mor- 
gan, DiGello, Wiles, & Rivers, 2004; 
Wilbur, 2000). Some students use an 
assistive hearing device or cochlear 
implant, whereas others do not use 
amplification devices at all (Kent & 
Smith, 2006; L. J. Spencer, Tomblin, & 
Gantz, 2012). Some identify as cultur- 
ally Deaf (sometimes along with other 
ethnic and cultural identities) or phys- 
ically deaf, or as a person who is hard 
of hearing (see, e.g., Mcllroy & Stor- 
beck, 2011; Nikolaraizi & Hadjikakou, 
2006; Schick et al., 2013; Smiler & 
McKee, 2007). 

The above characteristics are inter- 
related, and are a result of individual 
experiences, family history and 
resources, and cultural influences. Stu- 
dents’ characteristics are tied not only 
to their family background, but also to 
where they obtain their K-12 educa- 
tion. Educational assessments are 
largely conducted within the context 
of school settings; deaf education has 
a long history that features strong ties 
to community resources, the impor- 
tance of school identity, and a move- 
ment toward inclusive and local 
education settings and away from res- 
idential schools (Mitchell, 2006). The 
educational experiences of deaf and 
hard of hearing students thus reflect 
the great diversity of personal and 
community contexts in which these 
students live (R E. Spencer & Mars- 
chark, 2010). 

This diversity of demographics and 
educational experience has direct 
implications for how students who 
are deaf or hard of hearing participate 
in large-scale assessment systems 
(Cawthon, 2007, 2009), as well as 
curriculum-based and psychological 
assessments (Braden & Hannah, 1998; 
Luckner & Bowen, 2006; Soukup & 
Feinstein, 2007). Because most tests of 


student academic achievement are in 
written English, administered either 
on paper or via a computer system, 
reading skills and literacy development 
are an important part of accessing the 
content of the assessment. English lit- 
eracy skills, particularly academic liter- 
acy, vary widely in students who are 
deaf or hard of hearing (Qi & Mitchell, 
2012). English literacy levels are an 
access concern for students who are 
reading significantly below grade level, 
which is the case for many students 
who are deaf or hard of hearing, partic- 
ularly if they did not have access to a 
full language model during early child- 
hood (Luetke-Stahlman & Nielsen, 
2003). (For in-depth discussions and 
different perspectives on the linguis- 
tic determinants of English literacy 
development in deaf and hard of hear- 
ing students, see Czubek, 2006; Easter- 
brooks, Lederberg, & Connor, 2010; 
Kyle & Harris, 2011; Mayer, 2007; 
Mayer & Akamatsu, 1999; Myers et al., 
2010; Padden & Ramsey, 1993; Paul, 
2006; Strong & Prinz, 1997; Wilbur, 
2000; Williams, 2012). Beyond reading 
as an access tool for test content, there 
are also some concerns about deaf and 
hard of hearing students’ access to 
classroom learning opportunities 
(Moores, 2010; Vetter, Lohle, Bengel, & 
Burger, 2010). For example, students 
who use an interpreter as a classroom 
accommodation may not have the 
same level of direct instruction as stu- 
dents who receive direct instruction 
from the teacher (Cawthon, 2001; 
Marschark, Sapere, Convertino, & Pelz, 
2008). Furthermore, students may not 
have access to the incidental learning 
that happens in a classroom between 
students or between the teacher and 
other students; this can reduce the 
knowledge base they can draw from at 
the time of the assessment. 

Prevalent assessment accommoda- 
tions used by students who are deaf or 
hard of hearing include some of the 
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same accommodations used by other 
students with disabilities, such as 
extended time, a dictionary or glossary, 
or a separate setting (Cawthon & 
Online Research Lab, 2006, 2007). 
Some of these accommodations func- 
tion to provide support for students 
whose literacy level may not be at 
grade level and/or who may not have 
had full access to the content of the 
assessment (Cawthon, 2009). How- 
ever, there are additional accommo- 
dations that are more specific to 
language characteristics of students 
who are deaf or hard of hearing, 
including (a) having an interpreter 
translate test directions, reading pas- 
sages, or test items, either using sign 
language, a signed system, or a read- 
aloud approach; and (b) allowing the 
students to respond using sign lan- 
guage and having their responses 
recorded by a scribe who back-trans- 
lates those responses into English. 
When an interpreter or other sign 
language-based accommodations are 
used, there are potentially significant 
implications for the role the resultant 
scores play in high-stakes decisions. 
Many states do not allow these kinds of 
scores to be aggregated with unaccom- 
modated (or other accommodated) 
scores, thus reducing their viability as 
facilitators of inclusive assessment prac- 
tices (Clapper, Morse, Lazarus, Thomp- 
son, & Thurlow, 2005). Previous studies 
(Cawthon, 2007, 2009, 2011) illustrate 
how the participation of deaf and hard 
of hearing students in standardized 
assessments in which these types of 
accommodations are used has poten- 
tially negative implications at both the 
student and school levels. At minimum, 
the test score is less likely to repre- 
sent what the individual student has 
learned, and it is quite possible that 
decisions about the student’s trajec- 
tory, teacher effectiveness, and school 
status may be not based on sound 


information (Cawthon, Leppo, Carr, & 
Kopriva, 2013). 

Although there is a growing body of 
literature on the effects of accommo- 
dations for students who are deaf or 
hard of hearing, there has not been a 
systematic analysis of results from 
across the field. The purpose of the 
present study was to conduct a quali- 
tative meta-analysis of the research 
literature and to make recommenda- 
tions for future research and practice. 
Because of the potential interaction 
between test, student, and accommo- 
dations factors, it was critical to exam- 
ine the literature from the perspective 
of both student and test factors in 
order to understand how they may 
contribute to differential outcomes of 
test accommodations on the resultant 
scores (Cawthon, 2009). Three research 
questions guided this analysis: 

1 . What do research findings in the 
extant literature say are the effects 
of test accommodations for stu- 
dents who are deaf or hard of 
hearing? 

2. What test-level factors are in- 
cluded in the extant literature on 
the effects of accommodations 
on test scores for students who 
are deaf or hard of hearing? 

3. What student-level factors are 
included in the extant literature 
on the effects of accommoda- 
tions on test scores for students 
who are deaf or hard of hearing? 

Method 

In compiling the literature review for 
the present article, we searched for 
published works that focused specifi- 
cally on empirical investigations of 
accommodations on tests of student 
achievement and outcomes for stu- 
dents who are deaf or hard of hear- 
ing. We searched four research 
databases: Psyclnfo, ERIC, Dissertation 


Abstracts International, and Educational 
Abstracts. These databases cover a 
wide range of both education and psy- 
chology journals and other publica- 
tions that are likely to have articles in 
this area. We searched these databases 
because they house two primary 
sources of empirical reports: scholarly 
journals and assessment research 
organizations such as the federally 
funded education research labs and 
university research centers. All research 
submitted to the journals undergoes a 
peer-review process that helps ensure 
study quality. We did not put a date 
range limit on the search; the oldest 
paper that met our criteria was pub- 
lished in 1985, and the majority had 
been published since 2000. 

Our search terms included accom- 
modations , , test* assess* modifica- 
tion (s), D/deaf, hard of hearing , 
hard-of-hearing, hearing loss, hearing 
impaired, and deaf and hat'd of hear- 
ing (or hard-of-hearing') . The asterisk 
(*) is used in many search engines as 
a completion term to capture varia- 
tions on the word. For example, test* 
will return articles with test, testing, 
tests, etc. This increased the coverage 
of the search term to all variations that 
fit the word stem. Although there are 
technical differences between the two 
terms, for the purpose of this review 
we shall use the terms tests and assess- 
ments interchangeably. Although the 
focus was on assessment accommoda- 
tions, there were also several articles 
that looked at instructional accommo- 
dation formats and the potential 
impact of different methods of pre- 
senting information on student test 
scores. We included these when the 
study design allowed for implications 
to be reasonably drawn for an assess- 
ment accommodation. 

Our inclusion criteria for the analy- 
sis for the present study included the 
following: 


366 


Volume 158, No. 3, 2013 


American Annals of the Deaf 




4 ) 


• articles with experimental, quasi- 
experimental, or correlational 
designs as at least part of the 
methodology 

• research that included students 
who were deaf or hard of hear- 
ing or were otherwise indicated 
as having a hearing loss or 
impairment 

• studies with statistically signifi- 
cant and/or insignificant findings 

• educational settings ranging from 
K-12 through postsecondary 

• discussion of accommodations 
used during assessment, with 
measures of knowledge gained 
or information retained 

• dissertation studies and pub- 
lished manuscripts 

• studies that identified student- 
level variables and group mem- 
bership 

The studies in our literature review 
focused on a range of different types of 
assessments, including intelligence 
tests, instructor- or researcher-devel- 
oped assessments, state standardized 
tests, and nationally administered 
assessments. The type of assessment 
used in each study is relevant to this 
discussion because it helps to provide 
perspective on the purpose of the test 
and how an accommodation (or an 
accommodation package) might influ- 
ence test outcomes. 

Exclusion criteria eliminated studies 

• with instructional accommoda- 
tions only, without measurement 
of student knowledge 

• with samples of students with 
disabilities as a whole that did 
not disaggregate findings for 
individuals who were deaf or 
hard of hearing 

• that considered accommoda- 
tions outside an academic set- 
ting (e.g., in an employment or 
mental health setting) 


• with participants younger than 
school age 

• that were published in a lan- 
guage other than English 

• that were done for a thesis and 
were not otherwise published in 
a peer-reviewed journal 

Policy analyses and articles that were 
theoretically based were also excluded, 
as well as manuscripts specific to alter- 
nate assessments, specific groups of 
other students with disabilities, or 
work with English Language Learners. 

Summaries of each of the included 
articles are provided in Table 1. The 
articles are organized chronologically, 
from the earliest to the most recent. 
Each summary consists of a brief 
overview that includes a description of 
the sample, measures, potential con- 
founds, accommodations used, covari- 
ates, effects, and outcome variables. 
Studies were reviewed separately by 
both authors; in its final form, the table 
reflects joint decisions made during 
the discussion process. 

Results 

The three research questions (What 
are the effects of accommodations? 
What are test-level factors? What are 
student-level factors?) provided an 
overall context for the present study. 
The categories and themes for discus- 
sion within the sections of the article 
were generated as part of our review 
process, informed by both the 
research literature and issues in appli- 
cation to practice. In our analysis, we 
found that the overall effects of accom- 
modations were mixed — a finding con- 
sistent with research reviews of 
accommodations with other student 
groups — and that the results de- 
pended greatly on the type of test and 
the type of accommodations used. The 
types of accommodations used in the 
reviewed studies reflect an overriding 
priority of providing access to English 


print for students who are deaf or hard 
of hearing. This priority is related to 
the educational context of the partici- 
pants; many of the students included 
in these studies received classroom 
instruction in a signed language, either 
via an interpreter in an inclusive set- 
ting or directly from the instructor in 
an education setting focused on stu- 
dents who are deaf or hard of hearing. 
However, despite the importance of 
matching accommodations to individ- 
ual student characteristics and needs, 
few studies disaggregated findings 
beyond broad categories such as 
“deaf” or “DHH” or “hearing.” As a 
result, student-level characteristics 
were sometimes less salient in the dis- 
cussion of findings, and it was often 
more difficult to draw conclusions 
regarding the fit between students, the 
assessment, and the accommodations. 
This is a significant limitation of the 
research literature and raises questions 
as to the applicability of these findings 
to practice. 

ASL as an Accommodation 

Although students who are deaf or 
hard of hearing use a range of accom- 
modations, differences between ASL 
(or signed) and English text presenta- 
tions of information are the priority 
focus in many of the reviewed research 
studies. Translation of test items into 
another language raises questions 
about the validity of the construct 
being measured by the assessment. 
Johnson, Kimball, and Brown (2001) 
looked at issues surrounding inter- 
preter quality and the student experi- 
ence of using an interpreter during the 
assessment process. The study was a 
comparison of ASL and Signing Exact 
English (SEE II), a visual signed sys- 
tem with strong parallels to English 
syntax and phonology that is some- 
times used in classroom settings. In 
this study of student performance on 
a statewide math assessment, Johnson 
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Table 1 

Research on Accommodations and Students Who Are Deaf or Hard of Hearing 


Authors (date) 

Sample 

Test type 

Accommodation 

Outcome measure 

Mowl (1985) 

32 hearing impaired middle 
school students 

Statewide learning and 
literacy skills 

Item modifications to control 
syntax and vocabulary 

Math and reading 
scores 

Sullivan & Schulte 
(1992) 

368 DHH students 

Intelligence test 

ASL interpreter vs. signing of 
test items 

Factor structure; 
scores on test and 
subscales 

Maihoff et al. (2000) 

20 DHH students 

Standardized math 
achievement test 

Signed vs. written responses 
to test items 

Test scores 

Johnson et al. (2001) 

12 classrooms of students 

Statewide math test 

Signing Exact English (SEE II) 

Test scores 

Marschark et al. 
(2005) 

College students 
Experiment 1: 187 DHH, rest 
hearing 

Experiment 2: 20 interpreters 
Experiment 3: 20 DHH, 10 hearing 

Instructor-developed 
science test 

Video vs. live interpreting of 
material 

Test scores 

Marschark et al. 
(2006) 

College students: 105 DHH, 
22 hearing 

Instructor-developed math 
test 

ASL presentation of lectures; 
interpreter experience and 
familiarity 

Test scores 

Ansell & Pagliaro 
(2006) 

Early elementary: 59 Deaf 
students in schools for the deaf 

Researcher-developed 
word problems 

ASL interpretation of items 

Problem-solving 

strategies 

Wolf (2007) 

Elementary and middle school 
mainstreamed settings: 62 DHH 
students 

Stanford Achievement 
Test: Reading and 
Language (Mitchell, Qi, & 
Traxler, 2007) 

Extended time, teacher 
clarification, re-reading of 
directions, scheduling, 
simplified language in direc- 
tions, signed or read written 
directions 

Test scores 

Anderson-lnman 
et al. (2008) 

Middle and high school: 9 DHH 
students 

Instructor-developed sci- 
ence test 

Captions vs. expanded 
captions 

Gain scores from 
pretest to posttest 

Stinson et al. (2009) 

High school: 48 DHH students 
College: 48 DHH students 

Instructor-developed 
history test 

Lecture with speech-to-text 
vs. interpreting/note-taking 
services 

Student retention 
of test material 

Marschark et al. 
(2009) 

College students 

Experiment 1 : 20 DHH, 29 hearing 
Experiment 2: 23 DHH, 24 hearing 

Instructor- developed 
science test, both multiple 
choice and sentence 
completion 

Print vs. ASL/English pres- 
entation of material 

Test scores 

Russell et al. (2009) 

98 Middle and High school DHH 
students 

National Assessment of 
Educational Progress 

Human vs. avatar signer 

Test scores 

Steinberg et al. (2009) 

Fourth- and eighth-grade students: 
1 ,000 DHH vs. more than 1 million 
with other disabilities or English 
Language Learners 

State standardized math 
assessments 

Accommodated vs. not 
accommodated 

Item difficulty 

Hoffman & Wang 
(2010) 

First-grade students: 2 DHH 

Modified test item 

ASL graphics 

Parent and teacher 
reports of student 
engagement 

Cawthon, Wurtz, & 
Online Research Lab 
(2010) 

414 K-12 teachers and adminis- 
trators of DHH students 

State or district standard- 
ized assessments 

Extended time, test direc- 
tions interpreted, test items 
interpreted, test items read 
aloud 

Participant reports 
of accommoda- 
tions use 

Cawthon, Winton, 
et al. (2011) 

Fifth- through eighth-grade 
students: 64 DHH 

Math and reading items 
from state test 

DVD of ASL presentation 
with booklet vs. paper and 
pencil alone 

Test scores 
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et al. found that there were translation 
issues, specifically around items that 
had visual components, as well as 
open-ended questions that did not 
have specific multiple-choice responses. 
These findings emphasize the impor- 
tance of item type and content when 
decisions are being made about the 
feasibility of ASL as an accommodation 
for high-stakes assessments. 

Even with these cautions about dif- 
ficulty in language translation, findings 
largely point to a minimal impact of 
ASL as an accommodation on result- 
ant test scores. Maihoff et al. (2000) 
had inconclusive findings because 
their sample had only 20 students; 
also, there was not significant power to 
detect differences between the two 
test samples, and the samples were not 
measured with a similar assessment. 
Cawthon, Winton, Garberoglio, and 
Gobble (2011) also looked at the 
effects of ASL versus written text on 
reading and math test items taken 
from a state assessment (but not 
administered as part of the larger 
annual assessment). This study also 
did not find a significant effect of 
assessment format on student test 
scores. Interestingly, two additional 
factors, exposure to ASL in the class- 
room and student proficiency in read- 
ing, were both significant positive 
predictors of student test scores (over- 
all on both ASL and written formats). 

Marschark et al. (2009) looked at 
the effects of print versus “in the air” 
presentation of passages on the test 
performance of deaf and hard of hear- 
ing students and hearing students on a 
classroom science unit assessment. 
What makes this study unique is that it 
included both types of students and 
focused on the effects of the “in the 
air” format versus a printed English 
version of the materials. Deaf and hard 
of hearing students had ASL for their 
“in the air” format, and hearing stu- 
dents had auditory presentations of 


the passages. Across the board, all stu- 
dents had higher test scores on the 
print version of presentation than on 
the “in the air” format. This finding 
emphasized the strength of the print 
version of the materials over auditory 
or visual learning of the test content. 
In a follow-up experiment, Marschark 
et al. looked at student output on a 
writing passage assessment with “in 
the air” versus written response condi- 
tions for both student groups. Deaf and 
hard of hearing students scored higher 
on this assessment when they wrote 
out their responses in written format 
than when they responded “in the air” 
using ASL. Taken together, these stud- 
ies suggest that ASL as an accommoda- 
tion may not play an instrumental role 
in increasing access to written assess- 
ments for college-level students who 
are deaf or hard of hearing. 

Computer-based assessments in- 
crease the options that may be available 
for video-based or digitally generated 
accommodations. Russell, Kavanaugh, 
Masters, Higgins, and Hoffmann (2009) 
investigated the effects of an avatar 
signer versus a video-recorded human 
ASL interpreter on student perform- 
ance on the National Assessment of 
Educational Progress math assess- 
ment. Test scores from the NAEP are 
an alternative to state assessments as a 
tool for understanding the effects of 
accommodations on a nationally rec- 
ognized assessment. Students in this 
study were in grades 8-12. Although 
there was no paper-and-pencil com- 
parison format score for this assess- 
ment, students did have access to 
the print version of the item during 
the assessment. Students performed 
equally well on the NAEP with the 
avatar and video-recorded signer, a 
finding that suggests that the ASL 
presentation did not vary depending 
on its source. However, it is unknown 
if the students scored higher or lower 
using the ASL accommodation than 


they would have using only the English 
print version of the test. 

Finally, one study used an intelli- 
gence test in its investigation of the 
effects of assessment accommoda- 
tions on resultant scores. Sullivan and 
Schulte (1992) examined the differ- 
ence between ASL interpretation and 
signing by the test administrator. The 
focus of this study was on the factor 
structure of the Wechsler Intelligence 
Scale for Children-Revised intelligence 
test and subscales (Vocabulary, Infor- 
mation, Comprehension, Similarities, 
Arithmetic, Digit Span, Object Assem- 
bly, Block Design, Picture Completion, 
Picture Arrangement, Coding). This 
study is of interest both in that it ana- 
lyzed variables at the test, student, and 
family levels on overall outcomes and 
in that it looked at variations in how a 
visual representation of the items 
might affect the scores. For example, 
children with parents who were deaf 
or hard of hearing had higher mean 
scores on all scales. As far as the effect 
of the different types of administra- 
tions (ASL interpreter vs. signing 
administrator), there were no differ- 
ences in factor structure or student 
profiles. There was also not an analysis 
of the interaction of student-level and 
family-level factors with the type of 
administration. This finding is consis- 
tent with that of Russell et al. (2009) 
that the type of visual modality did not 
appear to change the validity of the 
test score. 

Participant 
Developmental Stage 

Given the significant changes in lan- 
guage, literacy, and content-area 
knowledge as students move from ele- 
mentary through postsecondary expe- 
riences, it is possible that the effect of 
accommodations may interact with the 
age or grade of participants. Most stud- 
ies in our literature review examined 
student performance within a similar 
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age range, either in elementary or post- 
secondary settings. One exception, 
Stinson, Elliot, Kelly, and Liu (2009), 
investigated how speech-to-text accom- 
modations versus interpreted lectures 
affected performance on multiple- 
choice and sentence completion tests. 
In their analysis across two different 
age groups, high school versus college 
students, Stinson et al. found that 
material retention was consistently 
greater for both groups on the multi- 
ple-choice test. However, for the type 
of accommodations used during the 
lecture, high school students did bet- 
ter with speech-to-text than with an 
interpreter, whereas college students 
did equally well under both condi- 
tions. The findings of this study sug- 
gest that the older students may not 
have relied on the accommodation for 
access in the same way as the high 
school students, perhaps indicating a 
greater flexibility in using input from 
ASL and written English conditions. 

The analysis of test item format 
(multiple choice vs. sentence comple- 
tion) in Stinson et al. (2009) is in line 
with related research on the effects of 
linguistic complexity on assessment 
scores for students who are English 
Language Learners (Abedi, 2006; 
Cawthon, 2011, Cawthon, Highley, & 
Leppo, 2011) or students with other 
disabilities (Cawthon, Beretvas, et al., 
2012). Multiple-choice items instead of 
sentence completion tasks are not an 
accommodation per se, and are used 
on many standardized assessments. 
However, multiple-choice items have a 
lighter linguistic load and, at times, less 
cognitive demand than sentence com- 
pletion items, which perhaps makes 
them easier for students who may have 
more emergent levels of English liter- 
acy than those who are fully proficient 
(and at grade level; Abedi, 2011). It 
may be the case that the conditions in 
the study by Stinson et al. (multiple 
choice vs. sentence completion) were 


already measuring different cognitive 
tasks, and that the assumption that the 
constructs being measured were the 
same across assessment formats may 
mask or alter how the effects of the 
two accommodations formats can be 
interpreted. 

Ansell and Pagliaro (2006) addressed 
issues of varying item difficulty and 
variation in student ASL proficiency on 
a math assessment. The tests were 
administered in ASL, and the investiga- 
tion included measurement of the stu- 
dents’ age and ASL skill development 
as factors in the analysis. The outcome 
of interest in this study was the type of 
problem-solving strategies students 
used on items with different levels of 
difficulty. This is a unique study in that 
the focus was as much on the way stu- 
dents answered the items as on 
whether or not the items were solved 
correctly. The unit test used in the 
study was a useful tool for this analysis 
because it afforded the opportunity to 
look closely at those different types of 
problem-solving strategies in a way 
that a standardized assessment could 
not. In this study, higher student ASL 
skill levels were related to the use of 
more viable problem-solving strategies 
on the math test items. While results 
were not controlled for by other devel- 
opmental factors such as literacy level, 
and a comparable test format in written 
English was not used, this focused atten- 
tion on the impact of ASL language level 
on test outcomes is a unique and valu- 
able contribution to the field. 

Single Versus Package 

The majority of the studies in the pres- 
ent literature review looked at one 
accommodation at a time, either with- 
in the context of a larger investigation 
of the assessment process or as an 
independent variable compared with 
another single accommodation. How- 
ever, in reality, most students in pri- 
mary and secondary education receive 


more than one assessment accommo- 
dation, or a package of accommoda- 
tions. The studies that did look at 
more than one accommodation at a 
time varied in whether these were 
deliberately chosen or arose naturally 
from the sampling process. For exam- 
ple, Steinberg, Cline, Ling, Cook, and 
Tongatta (2009) investigated item dif- 
ficulty for deaf and hard of hearing 
students with and without accommo- 
dations. Accommodations were not 
parsed out individually, but grouped 
as a student having one or more 
accommodations versus a student hav- 
ing none. The results of the study 
showed relatively few items with 
greater difficulty for students with 
accommodations than for those with- 
out, or vice versa, across the items and 
the grades (fourth and eighth grades). 
This finding suggests that the accom- 
modation packages did not change the 
difficulty of the item, and thus that 
inferences about the test score are still 
valid. Such an “in situ” study is perhaps 
a more authentic approach to under- 
standing the impact of an accommo- 
dated assessment than an experimental 
design, in practice, because it allows 
for accommodations to have been 
selected for students based on their 
characteristics and access needs. 

Wolf (2007) also considered the 
effects of accommodations as a package 
instead of conducting an experimental 
investigation of a single accommodation 
compared with another (or none at all) . 
In Wolf’s study, students used a range of 
accommodations, including extended 
time, teacher clarification of the ques- 
tion, re-reading the directions, flexible 
scheduling, simplified language in the 
directions, or directions presented in a 
signed language or read to the student. 
It is important to note that all accommo- 
dations used were deemed not to po- 
tentially violate the construct of the 
item. In addition to accommodation 
use, both individually and as a pack- 
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age, Wolf included information about 
the degree of hearing loss as a mediat- 
ing factor in this analysis, but did not 
include measures of language pro- 
ficiency in the study. First, in the 
analysis of use, participants were 
likely to employ similar accommoda- 
tions together (e.g., accommodations 
for directions). Second, in analysis of 
the impact of accommodations on test 
performance, the results depended on 
the content of the assessment. One 
particularly interesting finding was that 
the level of hearing loss and use of 
accommodations did not interact in 
their effect on test scores. In other 
words, there was not a differential 
effect of accommodations by poten- 
tially different levels of access to spo- 
ken English. 

Availability Versus Actual Use 

A challenge when evaluating the 
impact of different test accommoda- 
tions is to know whether or not a stu- 
dent is using the accommodation as 
intended, and, subsequently, to make 
an accurate interpretation of its poten- 
tial effect on a test score. In a study by 
Anderson-Inman, Terrazas, and Slabin 
(2008), middle school students took a 
science test with either standard cap- 
tions or “expanded” captions. The 
“expanded” captions linked test con- 
tent to additional contextual materials, 
such as written definitions, pictures, 
or concept maps. There were no 
reported differences in gain scores on 
the unit tests between the two condi- 
tions (although the sample of nine stu- 
dents, split between two groups, was 
perhaps too small to capture meaning- 
ful differences with the use of a group 
design). However, students did show a 
preference for the expanded captions, 
a finding that represented a common 
theme throughout the present litera- 
ture review: that students often pre- 
ferred one accommodation over 
another even when their performance 


did not indicate differential impact on 
test scores (e.g., Russell et al., 2009). 
One of the considerations in the study 
by Anderson-Inman et al. was the fact 
that merely presenting the accommo- 
dation did not mean that the partici- 
pants necessarily accessed it. 

A similar question about accom- 
modations use arose in a study by 
Cawthon, Winton, et al. (2011). In this 
study, students were provided two sets 
of items matched on content area and 
difficulty, one set provided in ASL via 
video and one set in print. The ASL ver- 
sion also included the text of the test 
item, so it was not the sole means of 
gaining access to the content of the 
question. The study did not include 
measures of eye gaze so as to ascer- 
tain the actual use of the ASL compo- 
nent of the video (see Marschark et al., 
2005, on inclusion of eye gaze in study 
analysis). Students had the printed 
booklets with the test items, and if 
they felt that the ASL component was 
unnecessary, they could have read all 
of the items in both the ASL and Eng- 
lish print formats. In a sense, this is a 
realistic version of what test taking 
with an accommodation is like for stu- 
dents in the classroom setting. Unless 
the entire assessment is administered 
without a booklet or test items in writ- 
ten format, the student is likely to 
have the option of reading the item in 
addition to or instead of using an ASL 
accommodation. Other than meas- 
ures of student preference, student 
use is not controlled for in the empir- 
ical studies of accommodations with- 
out strict manipulation of the testing 
experience. 

Discussion 

Each of the reviewed articles has con- 
tributed to the growing knowledge 
base surrounding assessment accom- 
modations for students who are deaf 
or hard of hearing. While the field is 
still in its infancy, even after nearly two 


decades of research there are some 
important implications from the pres- 
ent review as educational assessment 
in the United States continues its shift 
toward standardization and the inclu- 
sion of all students. 

Student-Level Factors 

The first implication of the findings of 
the present review is that, in many 
cases, student-level factors made just 
as important a contribution to the 
research outcomes as variations in 
accommodations. At times, a student- 
level factor was as simple as whether or 
not the student chose (or was able) to 
use the target accommodation during 
the learning process as well as during 
the assessment. In other cases, a stu- 
dent’s specific individual characteris- 
tics, including age, receptive ASL skills, 
academic proficiency in the test con- 
tent area, and facility with written Eng- 
lish, had a significant effect on test 
outcomes. Yet the extent to which 
studies analyzed how student charac- 
teristics varied across accommoda- 
tions conditions was limited. Few 
studies included more than one vari- 
able in the analysis, and very little 
information related to etiology, degree 
of hearing loss, and experience with 
language and literacy was included in 
most designs. Although few studies 
looked at potential interaction effects 
of these student-level factors with 
accommodations use, these character- 
istics are an intrinsic part of how edu- 
cation professionals make decisions 
about what accommodations will be 
appropriate for students to use. It 
would appear that the examination of 
accommodations use by deaf and hard 
of hearing students now must move 
beyond research on changes in test 
scores and into examination of the 
mechanisms that may underlie some 
of the potential interactions between 
accommodations, test formats, and 
student characteristics. 
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Primary Versus Supporting 
Accommodations 

While there are primary accommoda- 
tions that are directly linked to creating 
access to the assessment content, 
there are times when the effectiveness 
of a target accommodation is facili- 
tated by another, less emphasized 
accommodation within a classroom 
context. During class periods, for ex- 
ample, there are times when the 
instructional style makes it difficult to 
take notes and watch an interpreter at 
the same time. In this situation, a note 
taker may be seen as a supplementary 
accommodation. However, the effec- 
tiveness of having an interpreter may 
be somewhat contingent on the avail- 
ability of a note taker who is able to 
provide information in an accessible 
format. None of the studies we 
reviewed specifically looked at tar- 
geted packages of accommodations, 
with primary and facilitated accommo- 
dations, other than to note that many 
students use packages (Steinberg et 
al., 2009) and that students who are 
deaf or hard of hearing are likely to use 
multiple types of accommodations 
(Wolf, 2007). Targeted research on how 
packages support either the instruc- 
tional or the assessment process, 
viewed holistically and with considera- 
tion of how the package of accommo- 
dations function together, would help 
illustrate some potential best practices 
in this area. 

Computer-Based Assessment 

The study by Russell et al. (2009) ex- 
amining avatar signers versus video- 
taped human signers embedded in a 
computer-based assessment adminis- 
tration represents an emerging and 
potentially significant area of research. 
A computer-based test format meets 
many goals in a large-scale, inclusive 
assessment process. First, it is efficient 
to implement and, once set up, holds 
the potential for quick, large-scale 


administration of tests for many stu- 
dents within a relatively short time. 
Second, it allows for tailored assign- 
ment of accommodations at the stu- 
dent level, and perhaps at the item 
level. If, as indicated above, math items 
that had a high visual component pre- 
sented challenges to maintaining the 
construct being measured when pre- 
sented in ASL, it would be possible 
simply to omit the signing component 
of that test item. Finally, computer- 
based assessment leverages the poten- 
tial of computer -adapted assessment, 
such that students receive items that 
match their skill level as they make 
their way through the test. This pre- 
vents students from receiving too 
many items that are far above their 
proficiency level — which can increase 
frustration and reduce motivation to 
complete the assessment — and allows 
students to more efficiently demon- 
strate their knowledge (engagement 
in the assessment as facilitated by ASL 
graphics, discussed by Hoffman and 
Wang, 2010). 

The current literature can inform 
choices about the format of a signed 
accommodation in a computer-based 
assessment. Both the Russell et al. 
(2009) and Marschark et al. (2005) 
studies show that different versions of 
an ASL presentation do not affect stu- 
dent performance. However, the find- 
ings of Russell et al. did show that 
students preferred the latter format to 
the former. Student preferences and 
familiarity with interpreters were also a 
part of the study by Marschark, Con- 
vertino, Sapere, and Seewagen (2006), 
but were found not to be a significant 
predictor of student performance. If 
the focus of discussion surrounding 
accommodations in a computer-based 
assessment is on differential effects of 
a video, live, or avatar signer, then the 
research literature will indicate that the 
specific format of the signing accom- 
modation is not the key issue in achiev- 


ing a valid test score. If the focus of the 
discussion is on the learning process, 
using computer-based signed accom- 
modations during the classroom expe- 
rience, there may be other variables 
and factors that will be important to 
include in empirical studies before 
drawing conclusions about the rela- 
tive effectiveness of different signing 
options in computer-based learning 
formats. 

Fairness 

The validity of the interpretation of 
an accommodated test score is very 
important in the setting of assessment 
policy, the assignment of accommoda- 
tions, and the use of accommodated 
scores in accountability policies that 
rate teachers, schools, and districts on 
their effectiveness. Related to validity is 
the desire for a test that is fair for all 
students, or that accurately measures 
what they know. Fairness is sometimes 
defined in the assessment literature as 
having equal levels of item difficulty 
between a referent group (in this case, 
students with access to spoken lan- 
guage) and students who are deaf or 
hard of hearing. The study of item dif- 
ficulty discussed in the present review, 
Steinberg et al. (2009), indicated a mix 
of results as to the relative difficulty of 
accommodated versus unaccommo- 
dated test items for deaf and hard of 
hearing students. However, there was 
not enough specificity in either the 
sample demographics or the accom- 
modations used to draw strong con- 
clusions about the relative fairness of 
accommodated items for deaf and 
hard of hearing test takers. In the 
larger conversation about the po- 
tential changes in item constructs 
and the accessibility of test items, 
it would be necessary to look at 
item difficulty for an ASL- or signed 
language-accommodated test for stu- 
dents who are deaf or hard of hearing, 
preferably with a range of student-level 
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factors discussed in the literature 
review for the present article. Yet valid- 
ity and fairness within accountability 
systems such as NCLB or teacher merit 
programs are only indirectly measured 
by item difficulty analyses. A compre- 
hensive study of student-level factors, 
accommodations, resultant scores, 
and their use would require an exten- 
sive database across multiple states 
and a consistent policy of accommoda- 
tions use. Although complex and far 
from complete, the Common Core 
Standards Assessments currently un- 
der development have the potential to 
render this question answerable (i.e., 
the question of the effects of accom- 
modations on student scores for deaf 
and hard of hearing students and the 
items’ accessibility for these students), 
or at least amenable to modeling using 
sophisticated statistical techniques 
(Beretvas, Cawthon, Lockhart, & Kaye, 
2012 ). 

Conclusion 

Large-scale, standardized assessment 
is likely to remain a foundational part 
of how the U.S. education system, for 
both K-12 and postsecondary levels, 
measures student proficiency and 
progress toward academic goals. Many 
of the issues raised in the present arti- 
cle have parallels for other students 
from diverse backgrounds, including 
students with other disabilities, Eng- 
lish Language Learners, students from 
migrant families, and students who 
have otherwise not had the opportu- 
nity to learn in high-quality instruc- 
tional settings. Research into the 
specific implications of accommoda- 
tions for students who are deaf or hard 
of hearing is largely inconclusive, at 
least if one considers the high stan- 
dards of evidence required for a “gold 
standard.” The studies discussed in the 
present review address many issues, 
yet, in many cases, only for a broadly 
defined student population and with- 


out replication across settings and sub- 
ject areas. For example, not much is 
known about the effects of accommo- 
dations for deaf and hard of hearing 
students who have additional disabili- 
ties, the range of quality in the imple- 
mentation of ASL accommodations, or 
the resultant implications for test 
scores in how decisions are made 
about student proficiency and achieve- 
ment. (For example, will the resultant 
score determine whether or not the 
student will receive a high school 
diploma?) 

Yet even with these grey areas, 
accommodated assessments are being 
used every day to make high-stakes 
decisions that affect the future of stu- 
dents, teachers, and schools. In what 
is perhaps good news for inclusive 
assessment policies, in no instance in 
the present review did it appear that 
having an ASL-presented version of the 
assessment made the test items easier 
for students using the accommoda- 
tion — a finding that minimizes con- 
cerns about test score validity. Future 
test systems, including those under 
development for the Common Core 
Standards, may afford further opportu- 
nities for increased access to test con- 
tent for students who are deaf or hard 
of hearing. For example, although in 
the past using a computer to take a 
test may have been considered an 
accommodation, current assessment 
reform efforts will make it the norm. 
The move toward computer-based 
assessments may make it possible to 
individualize accommodations to 
address student-level and test-level 
characteristics in ways not available 
with traditional paper-and-pencil- 
based tests. We hope that this litera- 
ture review motivates test developers 
and researchers alike to think critically 
about how assessment accommoda- 
tions can facilitate learning outcomes 
for students who are deaf or hard of 
hearing. 
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